5 Computational Features from Evolutionary Operation to Parallel Direct Search: Pattern Search Algorithms for Numerical Optimization

variables may bedevil pattern search methods if there is a high degree of nonlinearity. On the other hand, pattern search methods have been applied successfully to problems with as many as 256 variables. Mead simplex method to a non-stationary point. Direct search methods for uncon-strained optimization on either sequential or parallel machines. Figure 5: A selection of admissible patterns and their mapping onto a rational lattice. Pattern search methods admit a wide range of heuristics. Coordinate search cautiously varies one coordinate at a time, whereas evolutionary operation using two-level factorial designs or composite designs may evaluate f at as many as 3 p points before choosing the next step. More recent examples of pattern search methods exhibit a similar diversity of strategies. For example, parallel direct search (PDS) 6] was designed to exploit the strengths of multiprocessor computing environments. By simultaneously evaluating f at multiple points on multiple processors, additional evaluations can be obtained \for free," often accelerating the search by reducing the total number of iterations required. Opportunities for computational parallelism abound and a parallel implementation of a particular class of pattern search methods has been developed 20]. In contrast, model-assisted grid search 22] exploits methods developed for the design and analysis of computer experiments 15]. The idea is to construct approximations to f that can be used to predict a single trial step that is likely to realize simple decrease on the current value of f (x k). Thus, pattern search methods can also be quite frugal. 6 Recommendations Experience suggests that pattern search methods are often useful in the following situations: (1) Evaluation of f is inaccurate. Even when f is de-terministic, computationally induced errors can occur, e.g., when evaluating f involves iterating to an approximate answer. One common diiculty occurs when these inaccuracies result in high-frequency oscillations in the returned function values. In such situations, derivative-based methods are likely to become trapped in the oscillations and thus fail to identify larger basins of smaller function values. (2) The derivatives of f are either not available or not reliable. In particular, inaccuracies in the evaluation of f often render nite-diierence approximations to the gradient unreliable. (3) The function f is not smooth. Although the existing convergence analysis for pattern search methods assumes that f is continuously diierentiable, we have found that pattern search methods are often eeective in practice when the partial derivatives of f are not …